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This study assessed the long-term effects on teaching 
approaches, acquired knowledge, and attitudes regarding small group 
teaching exhibited by faculty members who attended a 2-day workshop 
on Small Group Teaching. Each participant in the study was a 
physician on the faculty of the McGill University Department of 
Family Medicine and had attended one of several workshops given by 
the Faculty Development Committee of the Department between the years 
of 1988 and 1993. Ten study participants were randomly selected from 
the 47 faculty members who attended, and ten control subjects were 
selected from. the remaining faculty. Objectives of the workshops were 
to develop small group leadership skills and greater awareness of 
group dynamics. Participants were monitored for 6 months to 5 years 
afterwards to measure changes in cognitive learning, teaching 
behaviors, and attitudes. Three distinct instruments were employed to 
collect data relating to the three areas of learning. Despite not 
achieving noticeable statistical differences on all the instruments, 
several interesting trends did emerge which stack up in favor of 
participants who attended the faculty development workshops. The 
experimental group exhibited more specific small group teaching 
behaviors and greater knowledge about small group teaching than the 
control group. The experimental group also expressed more 
appreciation of small group teaching and a greater interest in 
learning more about group dynamics and teaching techniques than did 
the control group. (Contains 22 references.) (PRW) 
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Abstract: 



Purpose: This study was designed to measure change in cognitive 
learning, teaching behaviours, and attitude 6 months to 5 years 
after a workshop on Small Group Teaching. Method: Study 
participants included 10 faculty members who had attended the 
workshop between 1988 and 1993, and ten control subjects. Over a 
three-month period, three distinct instruments were employed to 
collect data relating to the three areas of learning. Results: The 
experimental group exhibited more specific small group teaching 
behaviours and greater knowledge about small group teaching than 
the control group. Significant differences also existed in the 
areas relating to the use of and attitudes towards this teaching 
method . 
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Introduction 



J 

Faculty development programs have been used extensively in 
the health professions to promote a wide range of knowledge and 
skills in teaching^ research and administration. While faculty 
development activities have been described in the literature 
( Hitchcock , Stritter , & Bland, 1993), few studies have actually 
examined the long-term impact of such programs on participants. 
Evaluation research has focused more on the short-term impact of 
programs with high faculty involvement, particularly fellowships 
and short programs, eg. workshops (Stritter, 1983). A variety of 
quantitative and qualitative instruments have been used to 
measure the outcomes of these programs (Hitchcock, et al, 1993). 
They have ranged from forms that assess immediate participant 
satisfaction to short-term measures of cognitive learning, 
attitude change, and teaching behaviours. 

The tools used to evaluate short-term change in these three 
areas resulting from fellowships (Hitchcock, Anderson, Stritter, 

& Bland, 1988; McGaghie, Bogdevic, Reid, Arnt, Stritter, & Frey, 
1990; Bland, Hitchcock, Anderson, & Stritter, 1987; Hitchcock, 
Lamkin, Mydal, Clarke, & O'Connor Clarke, 1986) and workshops, 
have included immediate self-reports, interviews, and structured 
questionnaires (Levinson-Rose & Menges, 1981). Certain studies 
have focused on participant satisfaction at the end of a faculty 
development activity (Linder & Witteman, 1984; Stritter & Hain, 
1977), others on change in teaching performance through faculty 
self-ratings and students' assessments (Skeff, Stratos, & Bergen, 
1992; Skeff, Stratos, Berman, & Bergen, 1992). Few studies have 




used more objective assessments of performance by observing 
teaching sessions (Skeff, Stratos, Campbell, Cooke, & Jones, 

1986; Skeff & Stratos, 1985. Nor has research addressed the 
long-term effects of faculty development activities on teaching 
( Jedrychowski & Galligani, 1978; Nerup, Thomsen, & Vejlsgaard, 
1972; Sheets & Henry, 1984; Sheets & Henry, 1988). 

Since most studies have relied heavily on self-report data, 
and self-reports have been shown to have a poor correlation with 
what teachers actually do (Hartman & Nelson, 1992), the results 
need to be interpreted with caution. Such data may be rendered 
more robust if they were complemented with more objective 
performance-based methods which are better indicators of the 
impact of short programs on acquired knowledge and change in 
teaching behaviours (Sheets & Schwenk, 1990; Linn, Baker, & 
Dunbar, 1991; Sheets, 1985). As well, studies are needed to 
evaluate the long-term impact of faculty development activities 
on attitudes, cognitive learning, and teaching behaviours. To 
this end, a study was conducted to assess the long-term effects 
on teaching by faculty members who attended a two-day workshop on 
Small Group Teaching between 1988 and 1993. 

Method 

Subjects 

Twenty randomly selected subjects comprising two groups 
(experimental and control) took part in this study. Participants 
in the experimental group included 10 faculty physicians who had 
attended a workshop on small group teaching between 1988 and 1993 
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and were randomly chosen from a total of 47 workshop c >* 
participants. Three had attended the workshop in 1988, one in 
1989, four in 1991, and one in 1993. From the remaining faculty 
members (n=45), another 10 were selected at random for the 
control group. Subjects were matched for the number of years of 
teaching experience. 

Intervention 

All experimental group members had attended the two-day 
workshop on Small Group Teaching which has been offered yearly 
since 1988 by the Faculty Development Committee of the Department 
of Family Medicine at McGill University. The main objectives of 
the workshop were to develop small group leadership skills and an 
awareness of group dynamics . The two days were divided into 
short theoretical didactic sessions and small group sessions 
involving group discussions, hands-on practica, and role plays. 
The content and format of the workshop have remained stable over 
the last five years. At the end of the course, participants were 
asked to fill out an evaluation form. Overall, the workshop 
activities have been highly rated with 80-90% of participants 
finding them either "useful" or "very useful". 

Data Sources 

Three instruments were developed and administered to collect 
data for this study. The first was an observation grid used for 
assessing leadership skills such as advance planning, convening 
groups, creating an atmosphere for learning, focusing group 
activities, summarizing and evaluating, and observing group 
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processes . Participants were observed at the beginning of the 
study while they taught a group of 4-10 residents on a topic of 
their choice at a t^m meeting or in sign-out rounds at the end of 
a clinic. A trained observer rated the teaching in the span of 
four weeks. Scoring values were "Done" , "Not Done", and "N/A". 

The second instrument consisted of 10 cases/scenarios 
depicting specific teaching tasks all of which had been presented 
and discussed in the workshop. This was used to assess cognitive 
learning. Five questions referred to leadership skills; five 
questions focused on group dynamics. A short answer format was 
used in order to assess participants' recall of the material 
covered. Points were assigned for each acceptable answer adding 
up to a maximum possible score of 35. 

The third instrument was a structured questionnaire for an 
interview conducted twice; once as a retrospective pre- 
intervention and again as a post- intervention. For the 
experimental group, pre- referred to the participants' view of 
their teaching career before they attended the workshop; for the 
control group, it was their early years in teaching. "Post-" for 
both groups was the present. This interview explored 
participants' use of small group teaching, the assessment of 
their own leadership skills, commonly encountered problems and 
strategies used to overcome them, knowledge of group processes, 
and use of innovative teaching methods. These instruments were 
administered in the order that they are introduced above. 
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Analysis : ' vx 

Chi square analysis were carried out to compare within and 
between group differences in the observation data. A Mann- 
Whitney U teat was carried out to establish within and between 
group differences in total scores yielded in the scenarios 
followed by chi-square analysis to determine within and between 
differences on each of the 10 scenarios. Finally, chi-square 
analysis were conducted for comparing within and between group 
differences in the two sets of coded interview data. 

Results 

Five individuals (50%) from the control group and seven 
(70%) from the experimental group were observed teaching in a 
team meeting setting while the rest of the participants taught at 
sign-out rounds post-clinic. The observation data showed no 
significant differences either between or within groups for any 
of the behaviors. Only one item, "puts together points made by 
members", approached significance (p=.056) between groups. 

Another item, "use of open-ended questions", was used more 
frequently by the experimental group (p=.ll). In general, the 
experimental group did have more frequent "Dones" (149 vs. 131) 
and fewer "Not Dones" (30 vs. 50) as compared to the control 
group (p=.013). Of note was the number of "not dones" in both 
groups for the item "makes a final summary for closure", and 
"obtains feedback from the participants on the teaching session". 




6 



With regards to the scenario results, there were no 
significant differences in total scores in either group although 
the experimental received a mean rating of 11.85 on the Mann- 
Whitney U Test while the control group received 9.15 (p=.30). 
Chi-square analysis for each question yielded no significant 
difference within or between group, although here again the 
experimental group scored higher on eight out of the ten 
scenarios . 

The interviews yielded information on the practice and 
teaching experience of the study participants, on their use of 
small group teaching, and on their attitudes towards this 
teaching method. The mean number of years in practice was 7.8 for 
the control group and 7.2 for the experimental group (p=.79). The 
mean number of years in teaching was 5.8 for the control and 4.7 
for the experimental group (p=.58). Two members from the control 
group had taken a fellowship year in Family Medicine early in 
their careers. Answers to the questions on the meaning, use, and 
enjoyment of small group teaching, understanding of group 
dynamics, and use of innovative teaching methods, are summarized 
in Table 1. 

On the pre-intervention retrospective interview, 50% of the 
control and 60% of the experimental group defined small group 
teaching as a lecture to a small number of people, while almost 
all of the remaining members of each group considered it an 
interactive session. This pattern changed in the post- 
intervention interview with 80% of the control group and 100% of 




7 



the experimental group defining small group teaching as 
interactive. The level of satisfaction and enjoyment of teaching 
small groups also changed. In the pre-intervention 
questionnaire, 80% of the control and 60% of the experimental 
group claimed that they enjoyed this method whereas in the post- 
intervention, this changed to 100% for the experimental group. 

The percentage for the control group remained the same (p=.109). 

With respect to awareness of group dynamics, 20% of both 
groups stated that they had an awareness before the intervention. 
This changed to 30% for the control group and 100% for the 
experimental group after the intervention (p=.003). Finally the 
frequency of use of innovative teaching methods was initially 
claimed to be 40% for the control group and 10% for the 
experimental group. Methods used included role plays and 
videotapes. This changed to 70% for the controls and 80% for the 
experimental group in the post- interview (p=.42), and included 
models, quizzes, games, debates, and handouts. 

The difficulties encountered in small group teaching by fche 
both groups remained stable: issues of credibility, lack of 
preparation, and problems engaging the group participants and 
dealing with difficult members. Both groups demonstrated a 
slight change in strategies used to overcome these problems by 
moving from "taking over and lecturing" to involving the 
participants in the teaching. 

Only one study participant, from the control group, had 
previously taken a workshop on Small Group Teaching other than 
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the one offered at McGill. Seven members from the control group 
expressed interest in learning more about different teaching 
techniques or methods that could be used in small groups. On a 
scale of 1 (not useful) to 5 (extremely useful), the experimental 
group rated the workshop in the following manner: three as a M 3", 
five as a "4", and two as n 5 H . In retrospect, the most useful 
sessions offered in the workshop were those that dealt with 
different teaching techniques and methods and group dynamics. 



The Small Group Teaching workshop was designed to promote 
changes in cognitive learning, attitudes, and teaching 
behaviours. The study instruments were developed to evaluate 
whether such change had occurred and had been sustained over 
time. By choosing the experimental group from a series of past 
participants, it was possible to look at a time frame of from six 
months to five years. The interviews looked at retrospective-pre 
and post self-report data in order to assess changes in attitude 
towards small group teaching. This design was used to avoid the 
response-shift bias seen in traditional pre test designs (Skeff, 
et al, 1992; Bland, Reineke, Welch, & Shahady, 1979). The 
scenarios explored cognitive learning, while the observations 
allowed for an objective performance-based evaluation of specific 
teaching behaviours. 

There may be several reasons that account for not achieving 
statistically significant results on all of the instruments. The 
number of participants in the study was small. As well, two from 
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the control group had graduated from a fellowship programme in 
Family Medicine giving them some of the teaching skills that were 
covered in the workshop. This study could, however, be considered 
as a pilot project to test the three instruments prior to 
carrying out a larger scale study . 

Despite not achieving noticeable statistical differences in 
many of the analysis, several interesting trends did emerge which 
stack up in favour of participants who attended the faculty 
development workshops. The experimental group exhibited more of 
the teaching behaviours taught in the workshop. Cognitive 
learning on both leadership skills and group dynamics was also 
found to be superior in this group. The retrospective pre- and 
post-intervention interviews yielded interesting data on 
attitudes and use of small group teaching. The experimental 
group felt that greater change had taken place in their 
understanding of this method, their use and enjoyment of teaching 
in small groups, their appreciation of small group dynamics, and 
their use of innovative teaching methods. As well, the workshop 
was rated as useful to very useful by those who had attended. Of 
note is their appreciation for the sessions on group dynamics and 
teaching techniques which were areas that the control group 
expressed interest in learning more about. This information will 
be valuable in planning future workshops. 

Each instrument was developed to measure a different 
learning outcome. From the above trends, it would appear that: 
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1) attitudes towards small group teaching improved more in the 
experimental group, 2) learning at a cognitive level was greater 
in this group, and 3) specific teaching behaviours were exhibited 
more by those who had attended the workshop. Whether these 
findings can be attributed to the workshop or to years of 
experience cannot be answered; however, the lack of significant 
within group differences on the observation grid and the case 
scenarios for both groups would argue against this. Nor can the 
possibility of "contamination" of the control group by the 
experience of the experimental group be assessed. 

A study with a larger number of participants using refined 
instruments is needed to answer these questions. Nonetheless, the 
authors believe that the workshop did have an effect on the 
attitudes, teaching skills, and cognitive learning of the 
participants in the long-term. The strength of their workshop can 
be attributed to: the choice of a relevant topic and educational 
methods, the involvement of participants, the use of credible 
facilitators, and the practical two-day format. These 
characteristics have been described by other faculty development 
authors in their recommendations for effective programs for 
improving clinical teaching in the health professions (Skeff & 
Stratos, 1985? Bland & Stritter, 1988). 
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Conclusion 



Faculty development workshops have been used in the health 
professions to improve the quality of teaching of faculty 
members. Few studies have examined the long-term effect of these 
activities on attitude formation, cognitive learning, and 
teaching behaviours. The educational importance of studies such 
as this one rests in the contribution they make towards 
accumulating empirical evidence for the long-term impact of 
faculty development programs. Moreover, as is the case with this 
study, they can serve as pilot data to refine instruments used to 
conduct larger scale studies as follow-up. 
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